Got Online Assessment Integrity Concerns? Canvas New Quizzes Provides Solutions

Good afternoon. Good evening. How are you all? I'm from Texas. Forgive me, if you hear, keep hearing you all. So if you have been enjoying all the sessions, all the sessions are great. I have been in a couple of them.

So this is in between our dinner. And, I hope, you will enjoy it. I'll try my level best to, keep you engaged. So How many how many of you are faculty here? Okay. So that's a good number.

Number. I know I have been, in this, field working with faculty for over twenty years. And started online in two thousand one. So since then, till today, When I try to, you know, work with the faculty who have been teaching fully face to face, I keep consistently hearing the same answer when I ask what keeps you not to, you know, offer online course. The first answer and only answer probably, consistently I have been hearing is the you know, I'm not sure whether student are gonna are gonna cheat or not.

So cheating or some kind of plagiarism, is, I believe, the highest concerned, you know, of our faculty, moving from face to face to either blended or or or online. So a little bit about me is my name is Abusa Farbaseth. I go by AZ because some people find it hard to pronounce, so probably it's easy to say AZ. My, academic preparation is computer science. My, PhD is educational technology from University of North Texas.

Anyone from Texas Great. Okay. So I'm from North Texas. My colleague, Michael Rodriguez, is here too, who helped me also in this presentation. We work together.

So, little bit about Tarrant County College. So Tarrant County is one of the largest, county. We have two million people live in Turn County, and Fort Worth City itself has one million people. And so the Turn County College, we have six campuses, five, physical location, and one online campus And it is the only online campus, fully online campus, among the community college international. So this is this is a model for other, callers to follow.

It has been, most like six, seven years, online. We are, Hispanicly Hispanic serving, institution, HSI, if you are familiar with, thirty over thirty six percent of our population are Hispanic. And if you see this, data, so the forty three thousand total head count, non duplicate head count, is after the pandemic. The before the pandemic, it was fifty thousand. So you can, see the reality.

And I think the community colleges hit the hardest during the pandemic. Over ten percent. And this is, the picture. If we count the duplicate head counts, our student population is close to hundred thousand. So, it is a very hours, institution.

We have nine hundred full time faculty, over thousand adjunct, across six, campuses. And, so we have a team of LMS in the back end. However, in the front end to support the faculty, faculty professional development day to day, support Michael and myself, so two people, support the entire faculty, population. So this is a little bit about Turn County. So now, the department that we are called organizational excellence and development.

It has four pillars. Learning and development, which is the institution wide strategic development team. We have, learning and development One is for compliance. The other pillar is for teaching and learning. And Michael and I under the academic technology or educational, technology pillar, we comes to, directly providing support for for Canvas, as well as all different technology.

That, facilitates, teaching and learning, including the procurement, so in faculty, submit a, request for Neo technology adoption. We go through a process. So, I know we heard about downtime in the keynote session. So it's it's really hard for us to have some downtime because our sessions are continuously building up throughout the semester. We have already the fall and the spring semester build up because we have to communicate with the faculty so they can schedule their calendar.

When I return this weekend, Monday, the new faculty boot camp for two weeks. Will start, for all the new faculty that have been hired, throughout, throughout the sister. Having said that, let's get to the point today's session. So I'm gonna try to cover, I'll I'll share some statistics because we all love data and the statistics. And then I will, show you some features.

However, not the features or the function how to create different kind of quizzes in new quiz, but, to, specifically go to the settings that is directly related to, you know, how to, minimize, the plagiarism or the student, cheating part, not how to create different kinds of quizzes, type of questions so you can learn it from the in a tutorial. And I'll share some, experience that I have been working with the faculty, how to address the student seating in online. Course, so that would be part of my, best practice. Now, let's see how concerned you are about, This, phenomenon, this challenge that we all are So, let me so this is maintometer. If you go to mentee dot com and use this code, and then share your response to this question.

Okay. So the code is cimenti dot com, and then code is one five seven one one four five nine. Okay. So again, one five seven one one four five nine. Okay.

Alright. Alright. So, so majority of us are concerned about cheating in online exam. So now I would like to share what some data that I I have collected. See what they are saying.

So, Rodgar's University professor, Donald Mckei did a twelve years long survey, and the research findings I'm gonna share with you later this, slides. However, this came from an article golden gate express after the pandemic, that ninety three percent of instructor thinks, the student are more likely to cheat online than in person. So this is actually higher than it used to be before the pandemic based on the data that I have I have looked at. So it's it makes, things work And thirty three percent said they were using some type of proctoring to prevent it. So again, proctoring, cause cost money and not every institution is, in a, situation to, have those services.

That's why, you know, the Canvas features that, Canvas has done a good job in terms of integrating a lot of, features to minimize that. So this is the results of the survey that Rodgar's professor Donald Mcabe did. And you can see here that from twenty nineteen versus, twenty twenty two, it's like seventy or more seventy five percent, student cheating or the plagiarism, happened during the pandemic. And, unfortunately, it's predominant area of stem, unfortunately. Your things, those are the smartest student, that's what, what the reality.

And if I go further, this is what the statistics looks like. Five million questions posted in year twenty nineteen, and twenty twenty two. And highest is one point seven million in chemistry, and followed by physics. Within those are the smartest student. And you can see the real data on the right side of the table.

That's how gloomy it looks like. Imagine this chart in twenty seven eighteen. And how many of you are, familiar with shag? Okay. Not not everyone. Not everyone.

That's where the student go. And then post their assignment, their quiz, all kinds of essays and everything. And someone from that side, does the job for the student. And it's real. How real it is you can tell twenty seventeen their company was two hundred and fifty five hundred thousand work.

And then after the pandemic, twenty nineteen, twenty twenty two, five hundred and six four thousand. So, it went up almost like three fold, the revenue. I know it's for them it's good but think about our kids. What they were doing, during that, during that time. So I hope I didn't depress you So this is, something to look at as well.

Look at the schools. There are like top schools. In the nation, and they are having this issue as well. And there are many articles probably you have seen during the pandemic. So I just, you know, post like this too.

And also, there is, I love NPR. It's a not Texas public, radio. You can you can listen to it by, using the stream, they cover every, aspect of our life. So, these are the real thing. So that's why, you know, why do we meet our faculty? Some of them are really resistance.

Because they do believe that, you know, these things happen, and we do believe these things happen. But what do we do? Should we not use the technology? So we have concern, but we also need to be positive and try to find a way, to address those concern, because you cannot just run away from the problem. Right? So, we thank, Canvas that they have done an excellent job particularly, in the new quizzes, and I'm gonna show you what, specific feature that you can utilize. To minimize or address those concerns. So now I would like to also know how familiar you are about with Canvas.

Oh, did you guys already responded? Excellent. Wow, so advanced. Yeah. So great. So look at that.

It looks like Yeah. All over the place. So that's good. There are some expert here. Please, share your, expertise and, knowledge with all of our audience.

So I'm gonna go directly to the canvas and then, bring up the quiz. Okay. We have been telling now we need to show how to do it. That's how we can learn and practice and make a difference. So I'm gonna go to my course So in, in the quiz, tool, you can see the new quiz is you can identify by the highlighted rocket versus these are the classic ways.

So that's that's how you can tell which one is the new quiz and the, classic quiz. So we So this is where the tool is. So you click quiz, at Tarrant County, there are some faculty who still want to, use the classic. So we didn't, you know, replace yet. So we are running concurrently.

Both options, new and the classic, but we try to promote the new quizzes because of, of the advantage, some type of questions like, hotspot and other, which is not available in classic quiz. However, we do keep the, you know, those faculty who are still comfortable doing that. So I'm not gonna create because I already have So I'm gonna go open a quiz, new quiz, So when you open a new quiz, you see, the build, this is how you build a quiz, So you click on build. Okay. So this plus sign is how you create new question, type of questions.

If I click on that, so these are the types of questions that you can, create using the new quiz. I know that the hotspot is not available in, classic, may be ordering, and categorization. So these are, couple of new. However, the hotspot and ordering, I think, are the two mostly widely used. Lot of our, geology, biology, stem faculty, they use the hotspot, type of question and the nursing faculty.

So like I said, I'm not gonna go and, these are very a simple, to do what I am gonna focus on here. Okay. So, settings. So, if you go to the settings, to, apply different functions to minimize the Number one is, shuffle questions. So you can, have, by the way, based on my experience working with faculty for like twenty years, over twenty years, I found out that Probably, I'm not gonna throw out a number, majority of them will actually use, multiple choice question, right? Because if you use asset type or categorization or, short questions, so there is a last chance of you know, less success of, you know, cheating, those type of theme pattern.

However, Most of the faculty, they will use the, multiple choice questions. And I have been seeing more and more are using the publisher content, test bank, which, unfortunately, not a good, good practice because, a lot of questions are not higher order thinking questions. If you are not building your own questions and just use the publisher, that sometimes, minimum, you know, I I would say it's not, probably best way to, assess a student higher order thinking skill. However, so, so shuffling questions. If you, have any number of, question, you can, utilize that features.

And on within the question, you can have the shuffling, answer prices. That is another way, to make it a little bit harder. And One question at a time, that is also a good practice because if you, release all the questions, at once, they can take a, you know, snapshot of the question and then share it with other students. So it is good to have one question at a time, release. There is an option to, restrict the backtracking However, so if I do that, so, however, it depends on your question.

Yeah. How are you using proctoring with new quizzes to help with that It depends on, you know, how much, you want to spend on those third party, software. So it's also depends on how many students are using, how many courses are using. You can always integrate the third party tool on top of the Canvas features. Right.

Okay. Yeah. You can. You can use the responders and I can show you some example in my other quizzes. So that's not the session is about.

Session is about how to minimize using the canvas new quiz features. Okay. Those are not canvas features. Those are third party you can always integrate. On top of the quiz.

So backtracking, is one of the way you can, stop so that they have to answer their questions they are sure, they are, you know, done, and they can go forward. Access code. It has, advantage and disadvantage, you know, in terms of the online, fully online settings. I have found, one of the most effective way to set the time limit. I know it sounds like, well, how can we do that? If you are a faculty and if you are creating your own question, then you know how long it should take to answer a question.

You know, rather than using the, you know, the the publisher come and things like this. And then if you try to create a higher order thinking question, I worked ten year in health science curriculum means medical school, biomedical, nursing. They use case based multiple choice question. And their entire assessment of the life assessment, their certification, all of them are multiple choice. So they are heavily multiple choice, assessment type of curriculum.

They are student. I don't think they cheat. Medical college student or nursing, or they'll be able to get away with cheating. So it depends on how much, Well, it will be probably very, very, very, very, I I would say is, very lower percentage. It's not like I showed like, you know, the computer science and other other things.

However, my point is, if you are a faculty, and taking your time and making a multiple choice question using the case study or data And, you know, making it a higher order, it's hard to cheat. They are not gonna be able to succeed. Okay. Hey, Doctor. Crazy, it's a great talk.

I was wondering, like, with features, have you looked at, like, looking at also, like, things kind of, like, creating questions in a format where you can catch plagiarism by how similar answers are. Looking at how students, like, progress through the quiz, because I know the new quiz engine has a has a concept where it reports data in the background on student was looking at which question and the quiz at which time. Mhmm. And if you look at the data, the amount between students, the I bring this up. Yeah.

And my advisor at at Georgia Tech has worked on similar technologies. Yes. Yes. It does. If you look at the total time, and then look at the detail how much time each student take what question and what they were doing.

And as an LMS administrator, I can also see what browser they're using or they're doing. Yes. That's the back end. Yes. It is possible to find out, but it takes a lot of time, to dig you know, deep.

Well, I mean, so I I'm building in that tech company right now. Signhardt, we're in Start Valley in New York. Okay. And we've done a lot of this kind of research and we actually have a product at that point. We'll get rare incorrect answers and how the students have progressed through the quizzes to detect that And then when we deployed it at a Georgia Tech class of nine hundred and thirty five students, we talked to thirty seven students.

And after approaching those thirty seven students, they came Okay. So, there is another, is IP restriction. We did it usually for nursing student. Even though it's online, we bring them to the lab and then use the IP, range. So they have to log in to those computer to take the, take the quiz.

So again, you need infrastructure, you need large you know, lab and then monitor it. So, and this is possible as well. So restrict students, results view. This is also another way actually, you can minimize, and also being a faculty, like I mentioned about, you know, you spend a lot of time building higher order thinking and then suddenly these are all released and the student took, you know, picture and, things like this, then it kind of, discourage you while I spend all the time and then, you know, this happens. So what medical call as, professor does they have a review session, very particular window of time.

They have to come to the class and then they are gonna the instructor will release it, discuss, and then close it back. So it's not, independent the review is not, completely online. Because you have to understand a multiple choice question, a good multiple choice question, creation takes ten to twenty hours. I have seen it. I work with them.

Actually, I have done a two year long research on creating multiple choice question by the student, not the faculty. Including the distraction, all the distractor, they have to explain why a is correct, y b is not correct, c is not correct, d is not correct. So if you really invest time to create multiple choice question, you can assess higher order thinking, scale some faculty will not agree with that. But you can. However, so one more features that is, in the new quiz, I want to show you.

So these are just basic, but like I said, if you set a time, if you know there are hundred questions and how long it actually, for student to take, then right there, you can minimize a lot of cheating because they will not have time to, look around, talk to their friends, if they know, they know it. If they don't know it, they don't know it. So, it's better to find the right time that is actually really need to set for that particular exam. So the features that I would like to show you on top of that, those things is called, random block, using random block. Anyone used.

Okay. So I am I am glad that couple of you are here. So, how do you do that? So you go to the item bank. So, I'm gonna bring some quizzes. So I'm gonna bring a geology class So when we use random block, what it does? So let's say you have a question bank of, let's say you are taking an exam, making an exam out of five chapter, right, or the five topics.

So each topic or each chapter has, fifty question or hundred question. And you want to create, block of five or maybe two or three's. Okay. And then bring fifty question or hundred question. And then randomly released ten, twenty or whatever number, right? So you have hundred question from each block.

So if you have like, three blocks, so you bring three hundred questions, and out of each block, you release, let's say, thirty three thirty three thirty four, right, from each block. So the system will randomly release from each block thirty three thirty three, thirty four, and make up one hundred question, exam out of three hundred questions. And statistically, it is a highly, highly chance that if student a and student b are sitting together are gonna get the same question. So, statistically proven. So this is, one of the best way that you can minimize using random block.

So you build the question bank And then peak and choose what are the, how many question you are gonna pick from how many question bank. And then make up an exam, as well as, So in here, also you have some options. So there are types of question. Like, you have like five, six, seven types of question. And you want to, so this is the filter that you can, use.

So it has only multiple choice question. So let me go back and get another quiz, which has different types of questions. So you'll see what I'm trying to bring item bank go to the item bank. Clora player. I know there are some.

Okay. So you can see. So the filter options this one is not available in the classic quiz. So this makes your life as a faculty, easy because the other one is all or none. So you have, you know, like hundred question, you have to bring all of them, or you have to pick one by one.

But these filter, functions allow you to pick and choose. Okay. I'm gonna take, like, fill in the banks, multiple choice, and then true or false. And then on top of that, you can pick, like, how many questions that you want to, bring. So, let's see if I say gonna bring multiple choice and the fill in the blanks.

So it automatically, select those questions and then so many. Okay. And then you come here. So there are, let's say, to any question. And then you choose either all question or you can choose randomly release question.

So you can release out of ten or twenty. Let's say five. And then done. So if you use at least three, block, random block, that would be probably the good enough to, have an exam that will probably not releasing the same question at the at the same time. Yeah.

Quick question. So, it says it's one point for a question and for you You can edit. Uh-huh. Yeah. You can edit.

You can assign any number. For random block, for random block? Yeah. So it has been same. For random block. Yes.

It has to be the same. But if you wanna take out of the random block, you can add as a single question. So it's not that you can, you can make an exam combination of individual question. So you can make an exam by choosing individual question as well as the random, block. So it's not that if you use the random block, you cannot add individual.

No, you can. So it's up to you. So there are some faculty they do. So they do particularly, they have their own question. They create own questions.

And those are, the question they will, keep out of the random block. And then, you know, within the random they will have combination of publisher content as well as their own content. So you have varieties of questions. So, the question that you create from your experience teaching in the classroom, would be definitely different than you know, if you if if because student also see the, the question bank, you know, sometimes from the practice quote, quiz Okay. Do the do the students get the same five questions? That's no.

That's the purpose of using the random block. No. Unless you say all. Yeah. That's the purpose of free.

Different than the previous one then. It must be because I've I've tried this on the whole the quizzes. Mhmm. And, obviously, just got the same question. So, probably, you didn't pick and choose a number of question to release how many number of question to release out of how many question.

So I think old one does have, but old one does not have the filter options like, you know, It's like all are unknown when you bring to the exam, but it does have, the option to release out of the the bank, like, you know, fifty out of one hundred or ten out of fifty or whatever it does have. Another question, because I never taken a a quiz within the quizzes as a student, but what they were showing the other day is that you ten questions to go back to. If you have them only answering what question at a time, then assume they can't You know? They can still can? They Yeah. Yeah. Yeah.

Yeah. Yeah. You know, if you got to study for release one question at a time and go back Yeah. So that's what the features is, backtracking. But some faculty, they don't want to, restrict it because some good student, they do also, you know, want to take advantage of, they may just go on time.

All the question and then come back and see they have, you know, taking as long as you have the time. So I don't think why would you restrict too much. Again, unless, you know, there is there is enough time for them, them to go and look around and find, the source from the outside of the exam. For, restricting going outside of the canvas, are the third party like you mentioned, are the Respondas, Proctorio, there are so many, Proctoring, software, that out there, it depends on how much money you want to spend. I believe you can only randomize within one bank.

If you have a second bank, I think it's shown after the first bank of partners. So would you randomize between more than one bank? Of course. Yeah. You can bring any number of banks. Yeah.

You can you can you can also pick and choose the, item bank. And then on top of that, you know, different types of questions. My other question is how would how would tagging useful? I saw that tagging option. For the filter, there was a you tag it when you create the question. It's kind of a metadata.

That's how you identify, the tagging. You know, use different, from the question stem, you identify those tech. When you save it, you use those tech. And then you can actually search and find those, term, the tag term, and then bring the question as well by searching. So there are there are many ways you can actually, do that.

Is there a way to be, like, question variance? The way the current tab about before is if it's twenty five pressure question banks. Each Bank has three or four questions in it. Mhmm. Well, first of all, is there a way to do that without having one question bank and, if so, is there a way that that will be more scalable? Would would you repeat it? Yeah. So, we want We want the, we want everyone to get.

Same questions. Okay. I see. Especially this is sometimes we have four cell phones that are tied to. Right.

Right. I haven't, checked it. If you are using all questions, for all students, then I don't know how how it's gonna make it, not to collaborate. Other than you you put the time restriction. And on top of that, maybe third party, software not to But again, third party software will not, unless you, have the video option on So there are so many variations, right, even though within the third party, responders if you use without the monitor, then it's one price.

And then if you add the monitor another price, so and then it comes with the tech support. Somebody has to go if is any issues. You have to go back and find those things. Yes. Yeah.

If you're using the same questions, I think your best bets like he mentioned using the time feature and possibly random. That's really the best thing. Yeah. So you can do the, I hope statistically it will, make some differences in terms of releasing the questions because yours you are using the random block. So it has an impact rather than you bring just hundred question and use the features like, question randomization and the answer randomization.

I believe if you use the block, it will be, more effective than just bringing hundred question. Because of the, the statistical, design and the and the formula that will work behind the scene in the back end. Trade off is that where you're comparing to your salsa exams. Right. Exactly.

Yeah. Well, yeah, I it's nothing is actually perfect, in that in that terms. But one thing I have learned from faculty that, the value of teaching that you have been teaching for, so long. If you create your questions and then at least, you know, maybe twenty, thirty percent of your own question you add, that's that will definitely add values. That will definitely add values.

Because you're gonna spend time and and and and The question would not be typical that you get from the, question bank from the publisher or just recall question and things like this. So, in terms of this, I also would like faculty to actually put their, their time when you make an exam. I'm fine if you are using, for practice exam, low stick exam, but not for, high stake exam, right? You can have ten, you know, ten points, practice exam throughout the semester. That's okay. But if you are putting like fifty percent of the weight of the entire course, and then you use you know, all of hundred percent question from the from the question bank.

I think, yeah, that's probably not gonna help. And also, the assessment, different types of assessments. So if you are using the formatted assessment, it's fine. Don't worry about too much cheating because your intention is let them learn and then grow. And then when you test them, the final exam or a midterm exam, then you put all of your, you know, tools and your, your imagination and and the value of teaching, that's that's important, actually.

You have so much experience teaching ten, twenty, thirty years. And then, you use just publisher, question bank. I think, yeah, that's, that's also another reason for more cheating. I hope I don't often I didn't offend the faculty. Okay.

Any more questions? So there are, some folks, they are using the advanced feature, you wanna share some something. No questions? I think we are done. Right? Okay. Thank you all. Thank you so much.

Please go to your app. That's the very bottom. Take the survey. Otherwise, I hope you have a lovely dinner and enjoy our entertainment tonight and around the applause for our presenters. Thank you.